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EXAMINER'S ANSWER 



This is in response to the appeal brief filed 09 February 2009 appealing from the 
Office action mailed 05 September 2008. 
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(1) Real Party in Interest 

A statement identifying by name the real party in interest is contained in the brief. 

(2) Related Appeals and Interferences 

The examiner is not aware of any related appeals, interferences, or judicial 
proceedings which will directly affect or be directly affected by or have a bearing on the 
Board's decision in the pending appeal. 

(3) Status of Claims 

The statement of the status of claims contained in the brief is correct. 

(4) Status of Amendments After Final 

The appellant's statement of the status of amendments after final rejection 
contained in the brief is correct. However, The Final Office Action was mailed 05 
September 2008, not "9-5-09" as stated by Appellant. 

(5) Summary of Claimed Subject Matter 

The summary of claimed subject matter contained in the brief is correct. 

(6) Grounds of Rejection to be Reviewed on Appeal 

The appellant's statement of the grounds of rejection to be reviewed on appeal is 
correct. 

(7) Claims Appendix 

The copy of the appealed claims contained in the Appendix to the brief is correct. 

(8) Evidence Relied Upon 

US 2004/01 72255 A1 AOKI et al 09-2004 
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US 2004/0201710 A1 



UCHIHASHI et al 



10-2004 



US 2002/0191071 A1 



RUI et al 



12-2002 



(9) Grounds of Rejection 

The following ground(s) of rejection are applicable to the appealed claims: 
Specification 

The specification is objected to as failing to provide proper antecedent basis for 
the claimed subject matter. See 37 CFR 1 .75(d)(1 ) and MPEP § 608.01 (o). 

Correction of the following is required: Applicant claims a "computer-readable 
storage media" in claims 34-45, which is not defined in the specification. 



The drawings are objected to as failing to comply with 37 CFR 1 .84(p)(5) 
because they do not include the following reference sign(s) mentioned in the 
description: the description of figure 3 in the specification includes an element 13, 
which is not included in the drawing. Corrected drawing sheets in compliance with 37 
CFR 1 .121(d) are required in reply to the Office action to avoid abandonment of the 
application. Any amended replacement drawing sheet should include all of the figures 
appearing on the immediate prior version of the sheet, even if only one figure is being 
amended. Each drawing sheet submitted after the filing date of an application must be 
labeled in the top margin as either "Replacement Sheet" or "New Sheet" pursuant to 37 



Drawings/Specification 
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CFR 1.121 (d). If the changes are not accepted by the examiner, the applicant will be 
notified and informed of any required corrective action in the next Office action. The 
objection to the drawings will not be held in abeyance. 

Applicant is required to amend the specification as needed to include any 
drawing corrections. 

Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

This application currently names joint inventors. In considering patentability of 
the claims under 35 U.S.C. 103(a), the examiner presumes that the subject matter of 
the various claims was commonly owned at the time any inventions covered therein 
were made absent any evidence to the contrary. Applicant is advised of the obligation 
under 37 CFR 1 .56 to point out the inventor and invention dates of each claim that was 
not commonly owned at the time a later invention was made in order for the examiner to 
consider the applicability of 35 U.S.C. 103(c) and potential 35 U.S.C. 102(e), (f) or (g) 
prior art under 35 U.S.C. 103(a). 
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Claims 1-8, 14-29, 32-41, 44 and 45 are rejected under 35 U.S.C. 103(a) as 
being unpatentable over Aoki et al (US 2004/0172255 A1) and Uchihashi et al (US 

2004/0201 71 0A1). 
Claims 1 and 22: 

Aoki discloses a system for communication (abstract; p. 1 , par. 001 1 ). Aoki 
discloses a first set of sensing and rendering components arranged to cover physical 
movements of multiple individuals present in a first environment. Aoki teaches a 
plurality of sensing components, such as microphones (p. 3, par. 0047) and vocalization 
detectors (p. 3, par. 0047); and rendering components, such as visual displays (p. 12, 
par. 0157; fig. 1), headphones (p. 14, par. 0170, lines 1-11), and audio speakers (p. 3, 
par. 0046), to cover physical activities and/or presence (tracking users' physical 
locations: p. 13, par. 0168, lines 8-10) of the individuals present in the environment. 
Aoki explains that the sensing and rendering components are used for detecting 
conversational characteristics (p. 4-5, par. 0061), such as user's physiological reactions 
(p. 9, par. 0114), user's vocalizations (p. 9, par. 0115), user's physical activity (p. 13, 
par. 0168), gestures such as mouse-clicks, button pushes, or voice commands directed 
at a computer (p. 14, par. 0174) of multiple individuals present in a (first environment) 
conversational floor (p. 1, par. 0012, 0013). Aoki teaches a context-awareness system 
that may be combined with many kinds of physical sensor data and computer 
application data to make assessments of user activity; the system being capable of 
tracking user's physical locations within a building and compute which users are co- 
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present in a room (motion or presence detection), (p. 13-14, par. 0168). 

Aoki discloses a second set of sensing and rendering components arranged to 
cover physical movements of multiple individuals in a second environment. Aoki 
teaches a system that enables groups and subgroups of people to communicate and 
carry on separate conversations within the context of a meeting (p. 1-2, par. 001 1- 
0014). Aoki explains that the invention can include additional types of inferential 
analysis that use input data other than those directly affected by conversation. The 
system may combine many kinds of physical sensor data and computer application data 
(p. 14, par. 0170) to make assessment of user activity for users who are co-present in a 
room and/or users who are not present (first and second environments) in the room (p. 
13, par. 0168-0169). 

Aoki discloses interest thread detector that uses the first and second set of 
sensing and rendering components to detect multiple communication interactions each 
involving a respective subset of the individuals present in the first and second 
environments and that maintains an interest thread for each communication interaction. 
Aoki teaches detecting multiple communication interactions involving subgroups of 
individuals present in the same or in different environments (p. 13-14, par. 0168-0169). 
Aoki teaches maintaining an interest thread for each communication interaction (p. 14, 
par. 0170). Aoki explains that the invention has a floor analysis module used to analyze 
data for conversational characteristics and that its results are used to determine users' 
activities, specify a set of mixing parameters, and communicate the mixing parameters 
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over a floor control path (p. 4, par. 0056, 0061 ; p. 5, par. 0061-0062; p. 6, par. 0072, 
0073, 0076). 

Aoki discloses communication provider that captures a set of media data from 
the sensing components and that combines the captured media data in response to the 
respective activities of each subset of the individuals and that communicates the 
combined media data to the rendering components. Aoki teaches a communication 
system and a method for receiving a plurality of communications from a plurality of 
communication sources; for mixing the plurality of communication for a plurality of 
outputs associated with the plurality of communication sources (p. 1, par. 0011-0013; p. 
2, par. 0014-0015). Aoki teaches combining the captured media data and 
communicating the combined media data to the rendering components (p. 14, par. 
0173-0178). Aoki teaches a plurality of modules and a "conversational floor 
configuration thread" for analyzing conversational characteristics of the plurality of 
individuals (p. 4, par. 0056-0058, 0061, 0062), and using the results of the analysis to 
control the floor controls and/or set of mixing parameters. Aoki explains that the floor 
configuration thread is invoked by the "invoke floor configuration thread" and that this 
thread is responsible for receiving results of the individual analysis modules, 
determining and selecting the configuration of audio sources from these results (p. 6, 
par. 0076). 

Aoki teaches analyzing data for conversational characteristics that can be 
determined from analysis of audio information or from physiological responses to the 
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conversation, such as responses measured by a biometric device and/or information 
from an eye-tracker device (par. 01 14) but Aoki does not specifically teach detecting 
physical movement of the individuals. However, Uchihashi discloses a system and 
method for computer assisted recording or capture of meetings or presentation events 
(abstract; p. 1, par. 0008). Uchihashi teaches a system having multiple cameras and 
sensors (p. 1 , par. 001 7, 001 9) for detecting physical motion of humans present in a 
conference (p. 2, par. 0022-0024, 0029; p. 4, par. 0046-0047). Uchihashi teaches 
displaying "candidate activity events", which are events of potential interest (interest 
thread) in the meeting (p. 5, par. 0051). Therefore, it would have been obvious to one 
ordinarily skilled in the art at the time the invention was made to modify Aoki's method 
of determining and analyzing the activities of multiple participants in a meeting to 
include Uchihashi's teaching of using sensors to capture physical movements of the 
participants because as Uchihashi says, the system follows the flow of the individuals' 
and/or the presentation's activities within the conference or meeting, and also because 
delays caused by camera switching latency is reduced or eliminated when the system 
captures only those activities that are of real interest to the meeting. 

Aoki/Uchihashi teaches combining captured media data in response to captured 
data in response to respective user's activities, and indicated by the user's shared 
communication (page 14, par. 0169), conversational characteristics (page 14, par. 
0170), user's gestures (page 14, par. 0174), and other captured data such as user's 
physical activities (page 13, par. 0168). 
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Claims 2, 23 and 35: 

See claim 1 . Aoki teaches selecting respective subsets of the first and second 
set of sensing and rendering components for use for each communication interaction 
(interest thread) or user's activities. Aoki explains that a meeting can have subgroups of 
people who carry on separate conversations, and that each subgroup can maintain 
awareness of the primary group conversation (p. 2, par. 0038). Aoki teaches a 
conversational floor wherein users are enabled to have side conversations in the 
conversational environment (p. 6-7, par. 0081-0082; p. 9-10, par. 0120). 
Claims 3, 24 and 36: 

Aoki teaches activities including speech levels of the individuals (user's 
vocalization) involved in the communication interaction (p. 3, par. 0035; p. 9, par. 0115, 
0118). 

Claims 4. 25 and 37: 

Uchihashi teaches activities including gestures by the individuals involved in the 
communication interaction (p. 3, par. 0035). 
Claims 5. 18. 20. 26 and 38: 

Uchihashi teaches activities including movements by the individual involved in a 
respective interest thread (p. 2, par. 0022-0024, 0029; p. 4, par. 0046-0047). 
Claims 6. 27 and 39: 

Uchihashi teaches capturing and storing location information about identified 
objects and individuals within the conference room (p. 3, par. 0031 ; p. 4, par. 0043). 
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Claims 7, 28 and 40: 

Aoki teaches refining captured data (p. 3, par. 0048). Uchihashi teaches 
processing the raw sensor information (p. 2, par. 0021; p. 4, par. 0045; p. 5, par. 0051). 
Claims 8, 29 and 41: 

Aoki/Uchihashi teaches storing captured data (Aoki: par. 0047; Uchihashi: par. 
0022, 0031, and 0033). 
Claims 14-15. 32-33 and 44-45: 

See claim 1 . Aoki teaches capturing communication interactions (interest 
threads) of conversational activities of different groups of individuals (main conversation 
and side conversation) creating in this way interest area for separate detected activities 
(p. 6-7, par. 0081; p. 9-10, par. 0120, 0130; p. 13, par. 0168). 
Claims 16 and 17: 

See claim 1 . Aoki teaches communication interactions involving two or more 
individuals (p. 1 , par. 001 1 ) in two or more environments (p. 1 3-1 4, par. 01 68-01 69). 
Claim 19: 

Aoki/Uchihashi teaches movement pertaining to rendering devices (Uchihashi: p. 
2, par. 0020; p. 3, par. 0035). 
Claim 21: 

See claim 1 . Aoki teaches participants may be in the same room or at a remote 
location (p. 13-14, par. 0168, 0170). 
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Claim 34: 

See claim 1 . Aoki discloses a computer-readable media that containing code (p. 
4, par. 0050, lines 1-4). 

Claims 9-13, 30, 31, 42 and 43 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Aoki/Uchihashi and Rui et al (US 2002/01 91 071 A1 ). 
Claims 9-13, 30, 31,42 and 43: 

Aoki/Uchihashi teaches tracking interactions pertaining to an individual or an 
object (p. 2, par. 0020, 0031 ) but it does not teach communication interactions 
pertaining to an artifact that changes over time. However, Rui discloses a method for 
recording and broadcasting meetings (abstract; p. 1, par. 0009) wherein communication 
interactions that pertain to an artifact, such as a whiteboard, are captured and stored (p. 
1-2, par. 0010). Thus, it would have been obvious to a person having ordinary skill in 
the art at the time the invention was made to modify Aoki/Uchihashi's method of 
capturing communication interactions to include Rui's teaching of capturing 
communication interactions that pertain to devices because the participants are enabled 
to share a workspace, such as a whiteboard, and capture annotations that can be 
viewed by all participants in the conference room and then later reviewed by those 
individuals who were or were not present in the meeting. 
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(10) Response to Argument 

A. Appellant argues (page 8, first paragraphs and second paragraph; page 9, lines 
1-2), "Aoki and Uchihashitio not disclose or suggest detecting multiple communication 
interactions involving the physical movements of multiple individuals in respective first 
and second environments and then combining media data in response to activities 
indicated by their physical movements as claimed in claims 1, 22, and 34... The 
examiner has stretched the limit of reason beyond its breaking point in applying the 
conversational floors of Aoki to the physical environments of claims 1, 22, and 34 by 
stating that Aoki teaches 'detecting conversational characteristics... of multiple 
individuals present in a (first environment) conversational floor... '...a conversational floor 
as taught by Aoki is not an environment as claimed... because a conversational floor as 
taught by Aoki is a sub-conversation carried on a shared communication 
channel... whereas an environment as claimed. ..is a physical environment having 
sensing and rendering components that cover the physical movements and activities of 
multiple individuals present in the environment." 

In response, Aoki discloses detection of "presence/absence" of a user's 
vocalizations (audio capturing; page 2, par. 0035-0037); video information, biometric 
information, eye-tracking information (page 4, par. 0061); a conversational floor 
configuration thread for analyzing conversational characteristics (page 5, par. 0062, 
0076-0077); heuristic analysis performed to keep the conversational floors stable 
without interfering with the ability for the users in the conversational environment to 
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have side conversations (pages 6-7, par. 0081); turn taking analysis thread that 
determines when users are taking turns talking with each other (page 10, par. 0127); a 
determining feature from a voice activity detector procedure that indicates that two 
speakers are in the same conversational floor (one physical environment); and indicates 
when two speakers are not in the same conversational floor (two physical 
environments); a shared likelihood of shared floor procedure that determines the 
likelihood from the quantitative values returned from the feature that "this user" and the 
"other user" share a conversational floor (page 10, par. 0132). Aoki discloses that the 
invention can be augmented with a context-awareness systems that combine many 
kinds of physical sensor data and computer application data to make assessments of 
user activity. Aoki explains that the context-aware system is capable of tracking users' 
physical locations within a building and can computer which users are co-present in a 
room, can assess copresent users' vocalizations as having a high probability of being 
directed at each other as opposed to being directed at remote users (page 13, par. 
0168). Aoki discloses that the invention allows multiple users of a shared 
communication environment to automatically establish conversational floors that allow 
different groups of users to converse while still having the capability of being aware of 
other conversations (page 14, par. 0169). Aoki explains that other known techniques 
can be used to extend the invention to shared communication environments in which 
not all users are necessarily in remote locations and associated with a personal input 
device (such as microphones) and a personal output device (such as headphones). 
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The physical environment (such as rooms within a home or an office building) can be 
augmented with sensors that track individual users as well as directional beam-forming 
arrays consisting of multiple microphones so that vocalizations of each tracked user can 
be captured as a separate stream (page 14, par. 0170, 0178). 
Uchihashi was used for its specific teaching of detecting physical movement of the 
individuals present in at least two separate physical environments. Uchihashi discloses 
a graphical meeting capture controller user interface for displaying images from one or 
more room cameras and other image sources including a computer display, a video 
tape recorder/player, a satellite feed, etc. (page 1 , par. 0019). Uchihashi discloses 
intelligent image sensors that determine physical motion by comparing two successive 
image frames (page 2, par. 0022); using color to indicate detected sound activity and 
detected physical movement (page 2, par. 0029). Uchihashi discloses that physical 
motion activities in a room (physical environment) can be captured using cameras 
(pages 4-5, par. 0047, 0051). Therefore, Aoki/Uchihashi discloses detecting multiple 
communications interactions involving the physical movements and activities of multiple 
individuals in respective first and second activities indicated by their physical 
movements as claimed in claims 1, 22 and 34. 

(11) Related Proceeding(s) Appendix 

No decision rendered by a court or the Board is identified by the examiner in the 
Related Appeals and Interferences section of this examiner's answer. 
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For the above reasons, it is believed that the rejections should be sustained. 
Respectfully submitted, 

/X. L. Bautista/ 

Primary Examiner, Art Unit 2179 

Conferees: 
Ba Huynh 
/Ba Huynh/ 

Primary Examiner, Art Unit 2179 
/Weilun Lo/ 

Supervisory Patent Examiner, Art Unit 2179 



